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Abstract. We address the problem of surface inpainting, which aims to fill in 
holes or missing regions on a Riemann surface based on its surface geometry. 
In practical situation, surfaces obtained from range scanners often have holes 
or missing regions where the 3D models are incomplete. In order to analyze the 
3D shapes effectively, restoring the incomplete shape by filling in the surface 
holes is necessary. In this paper, we propose a novel conformal approach to 
inpaint surface holes on a Riemann surface based on its surface geometry. The 
basic idea is to represent the Riemann surface using its conformal factor and 
mean curvature. According to Riemann surface theory, a Riemann surface can 
be uniquely determined by its conformal factor and mean curvature up to a 
rigid motion. Given a Riemann surface S, its mean curvature H and confor- 
mal factor A can be computed easily through its conformal parameterization. 
Conversely, given A and H, a Riemann surface can be uniquely reconstructed 
by solving the Gauss-Codazzi equation on the conformal parameter domain. 
Hence, the conformal factor and the mean curvature are two geometric quan- 
tities fully describing the surface. With this A- if representation of the surface, 
the problem of surface inpainting can be reduced to the problem of image in- 
painting of A and H on the conformal parameter domain. The inpainting of 
A and H can be done by conventional image inpainting models. Once A and 
H are inpainted, a Riemann surface can be reconstructed which effectively re- 
stores the 3D surface with missing holes. Since the inpainting model is based 
on the geometric quantities A and H, the restored surface follows the surface 
geometric pattern as much as possible. We test the proposed algorithm on 
synthetic data, 3D human face data and MRI-derived brain surfaces. Experi- 
mental results show that our proposed method is an effective surface inpainting 
algorithm to fill in surface holes on an incomplete 3D models based their surface 
geometry. 

1. Introduction. Surface processing is an important topic in computer graphics, 
computer visions and medical imaging. Examples includes surface denoising, surface 
inpainting, surface remeshing and so on. Amongst the various tasks in surface 
processing, the problem of surface inpainting, which aims to fill in holes or missing 
regions on a Riemann surface, is an important pre-processing step necessary for the 
geometric analysis of incomplete 3D shapes. Digital 3D surfaces containing holes 
are common. In practical situation, real-life objects are usually captured from range 
scanners. Despite the recent advance in the laser scanning technology, 3D models 
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obtained from even the most sophisticated acquisition devices are often incomplete 
due to occlusions, low reflectance, constraints during recording or self-occlusions 
of complicated geometry. This results in holes or missing regions in the obtained 
3D digital models. In order to analyze the geometric objects effectively or derive a 
visually appealing 3D models, developing an effective algorithm for filling in holes 
or missing regions on a surface is of utmost importance. 

In this paper, we propose a framework for filling in holes or missing regions on 
a Riemann surface using conformal geometry. The ultimate goal is to obtain a 
restored surface with holes filled based on the surface geometry, so that the ba- 
sic geometry of the surface can be smoothly patched. For this purpose, a good 
representation of the Riemann surface describing its geometry is required. In this 
work, we propose to represent the Riemann surface using its conformal factor and 
mean curvature. According to Riemann surface theory, a Riemann surface can be 
uniquely determined by its conformal factor and mean curvature up to a rigid mo- 
tion. Given a Riemann surface S, its mean curvature H and conformal factor A can 
be computed easily through its conformal parameterization. Conversely, given A and 
H, a Riemann surface can be uniquely reconstructed by solving the Gauss- Codazzi 
equation on the conformal parameter domain. Hence, the conformal factor and the 
mean curvature are two geometric quantities fully describing the surface. Given the 
X-H representation of an incomplete surface, the problem of surface inpainting can 
be reformulated into the image inpainting problem of A and H. Using the conven- 
tional image inpainting techniques, both A and H can be inpainted on the conformal 
parameter domain. A Riemann surface can then be reconstructed which restores 
the 3D surface with missing holes. Since the proposed inpainting model is based 
on two geometric quantities fully determining the surface, the surface holes can be 
effectively filled , which follows the surface geometric pattern. We test the proposed 
algorithm on synthetic data, 3D human face data and MRI-derived brain surfaces. 
Experimental results show that our algorithm can effectively inpaint surface holes 
based on the surface geometry to restore the incomplete 3D surface models. 

The contributions of this paper are two-folded. First, we consider the repre- 
sentation of a Riemann surface using its conformal factor A and mean curvature 
H. Given two scalar functions A and H, the associated Riemann surface S can 
be reconstructed through solving the Gauss-Codazzi equation numerically. Second, 
we propose a surface inpainting algorithm by inpainting the scalar functions A and 
H. The surface inpainting algorithm is based on A and H, which are two geomet- 
ric quantities fully describing the surface. Hence, the proposed surface inpainting 
algorithm can fill in surface holes based on the surface geometry. 

Our paper is organized as follows: prior work on related topics is presented in 
section 2. The basic mathematical theory is discussed in section 3. In Section 4, 
the details of our proposed surface representation using conformal factor and mean 
curvature are discussed. The detailed algorithm for surface inpainting is described 
in Section 5. In Section 6, the numerical algorithms are summarized. Experimental 
results are reported in Section 7, and some conclusions and future work are discussed 
in Section 8. 

2. Previous work. In this section, we will review some relevant works closely 
related to our paper. 

The problem of surface inpainting was inspired by 2D image completion, which 
has been widely studied by different groups. Different variational models have been 
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proposed [1, 2, 4, 5, 6]. The basic idea is to diffuse the image intensity into the in- 
painting domain from its surrounding regions based on various regularizations of the 
solution. For example, Chan et al. [1, 2] proposed the total variation (TV) models 
for image inpainting. Euler elastica model [4, 5, 6] has also been proposed, which 
is essentially a variational model with a regularizing term containing higher-order 
derivatives. Other examples include the active contour model based on Mumford 
and Shahs segmentation [10], the inpainting scheme based on the Mumford- Shah- 
Euler image model [3], the inpainting model with the Navier-Stokes equation [7], 
and wavelet-based inpainting algorithms [8, 9]. For a detailed introduction to image 
inpainting, we refer the readers to [4]. 

3D surface processing has also been extensively studied. Different surface de- 
noising algorithms have been proposed. For example, surface smoothing by mean 
curvature flow or Laplace smoothing have been proposed [11, 12, 13], which are 
effective for removing oscillations on noisy surfaces. In order to preserve ridges and 
sharp corners of the surfaces, different second order anisotropic surface diffusion 
models have been considered [14, 15, 16, 17], which minimizes the weighted surface 
area. Higher-order isotropic or anisotropic flows based on minimizing the surface to- 
tal curvature or weighted total curvature have also been discussed [18, 19]. Besides, 
various surface inpainting algorithms to fill in surface holes have been proposed 
[20, 23, 22, 24, 25, 21]. A common approach is to fill in the occluded regions with 
surface patches, which smoothly attach to the boundary vertices of the surface holes 
[23, 22]. For example, Clarenz et al. [23] proposed to fill in surface holes by mini- 
mizing the Willmore energy functional. The surface patches obtained is guaranteed 
to satisfy the continuity properties. Davis et al. [22] proposed to extend a signed 
distance function, which is initially defined only on domains close to the known 
surface, to the complete space using volumetric diffusion. The surface can then be 
completed, even for non-trivial hole boundaries. Later, Caselles et al. [20] proposed 
a level set variational framework based on the minimization of the L p norm of the 
mean curvature for surface inpainting. The algorithm produces an inpainted surface 
with minimal surface area given the boundary constraints, although the restored 
surface may produce non-smooth global geometry. Most of the above algorithms 
do not incorporate or only partially incorporate the geometric information of the 
surface into their models. The inpainting result obtained usually does not follow 
surface geometric pattern, and thus the restored surface might often be unnatural. 
In this paper, we propose to inpaint surface holes through inpainting the conformal 
factor and mean curvature, which are two geometric quantities fully determining 
the surface. The inpainted surface can better follow the surface geometric pattern. 

The proposed surface inpainting model in this paper is based on the confor- 
mal structure of the Riemann surface. The problem of computing the conformal 
structure of a Riemann surface has been extensively studied, and different algo- 
rithms have been proposed [27, 28, 30, 35, 26, 29]. For example, Hurdal et al. 
[26] proposed to compute the conformal parameterization using circle packing and 
applied it to the registration of human brains. Gu et al. [28, 30, 29] proposed to 
compute the conformal parameterizations of human brain surfaces for registration 
using harmonic energy minimization and holomorphic 1-forms. Using conformal 
factor and curvatures, a shape index has also been proposed to measure geometric 
difference between hippocampal surfaces [32, 33]. Conformal structure has further 
been applied for geometric compression. In [31], the author proposed to compute 
the conformal structure using holomorphic one-form. The conformal structure is 
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then applied to estimate the edge lengths of the triangulation mesh. Together with 
the angle between triangular faces, the surface mesh can be reconstructed by solving 
a linear system. Hence, instead of 3 coordinate functions, we can store the surface 
using only the conformal structure and angles between triangular faces. 

3. Mathematical background. In this section, we describe some basic mathe- 
matical concepts relevant to describing our algorithm. 

3.1. Conformal structure of a Riemann surface. A surface S with a conformal 
structure is called a Riemann surface. All Riemann surfaces are locally Euclidean. 
Given two Riemann surfaces M and N. We can represent them locally as: 

</>m(xux 2 ) : R 2 M C R 3 and <t> N (x 1 ,x 2 ) : R 2 N C R 3 (1) 
respectively, where (xi,x 2 ) are their coordinates. The inner product of the tangent 
vectors at each point of the surface can be represented by its first fundamental form. 
The first fundamental form on M can be written as 

ds M = ^2 9ijdx l dx J , (2) 

where g^ — • j^r ano ^ hJ ~ 1 5 2. Similarly, the first fundamental form on N 
can be written as 

ds 2 N = ^^gijdx L dx\ (3) 

where 9ij = • and ij = 1, 2. 

Given a map / : M — >• N between the M and N. With the local param- 
eterization, / can be represented locally by its coordinates as / : R 2 — ^ i? 2 , 
f(xi,X2) = (fi(xi, x 2 ), ^2(^1, X2))' Every tangent vectors v on M can be mapped 
(push forward) by / to a tangent vectors f*(v) on N. The inner product of the 
vectors f*(v{) and /*(^)), where v{ and v~2 are tangent vectors on M, is: 



f*(ds 2 N )(v ll v 2 ) := < f*(vi),f*(v 2 ) > 



1,3 



(4) 



dh dfj 



i,j m,n 



Therefore, a new Riemannian metric f*(ds 2 N ) on M is induced by / and ds 2 N , 
called the pull back metric. We say that the map / is conformal if 

t{ds 2 N ) = \{x 1 ,x 2 ) 2 ds 2 M (5) 
A is called the conformal factor. It is one important geometric quantity describing 
the Riemann surface S. 

A parameterization cp : R 2 — >• M is a conformal parameterization if ip is a con- 
formal map. 

Intuitively, a map is conformal if it preserves the inner product of the tangent 
vectors up to a scaling factor, called the conformal factor, A. An immediate conse- 
quence is that every conformal map preserves angles. Figure 1 gives an example of 
conformal parameterization of a human face. (A) shows a human face. It is confor- 
mally parameterized onto the 2D rectangle as shown in (B). We map the texture 
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Figure 1. Example of conformal parameterization. (A) shows a 
human face. It is conformally parameterized onto the 2D rectangle 
as shown in (B). We map the texture in (B) onto the surface, as 
shown in (C). Note that the right angle structure of the checker- 
board texture is well-preserved. (D) shows the histogram of g\2 (or 
921) of the Riemannian metric. 



in (B) onto the surface, as shown in (C). Note that the right angle structure of 
the checkerboard texture is well-preserved. (D) shows the histogram of gi2 (or #21) 
of the Riemannian metric. It is concentrated at 0, meaning that the mapping is 
conformal. 

3.2. Curvatures of a Riemann surface. We next briefly describe the concept 
of curvatures on a Riemann surface. Given a Riemann surface 5, the normal cur- 
vature n n in some direction is the reciprocal of the radius of the circle that best 
approximate a normal slice of the surface in that direction. For smooth surface, it 
can be computed from a 2 x 2 symmetric matrix W, called the Weingarten matrix, 
by: 

Kn = ( u ,v)w( U v ^)=(u,v)( e f f g )( U v ) (6) 

for any unit length vector (u, v) in the tangent plane of the surface. 

The principal curvatures, principal directions, mean curvatures and Gaussian 
curvatures can be defined by the eigenvalues and eigenvectors of W. Let k\, hi be 
the eigenvalues of W and d\, (I2 be the eigenvectors of W. Then, k\ and &2 are 
called the principal curvatures] d\ and (I2 are called the principal directions. The 
mean curvature H is defined as the average of the principal curvatures: 

H= k 1 +k 1 (7) 

The Gaussian curvature K is defined as the product of the principal curvatures: 

K = k 1 k 2 (8) 

With the conformal parameterization <fi : D C M? — >• S of S, the Gaussian 
curvature K and the mean curvature H can be computed easily: 

K = -^AlogA (9) 
where A is the conformal factor. And 
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H = ^sign(^)|A^| (10) 

where sign(0) = < ^ ) ^ l> and n is the (unit) surface normal. 

It turns out the mean curvature H together with the conformal factor A are two 
geometric quantities fully describing the surface S. 



4. Surface representation by conformal factor and mean curvature. In 

this section, we will describe how a Riemann surface S can be represented by its 
conformal factor A and mean curvature H. 

Every Riemann surface is locally Euclidean, and hence it can be locally parametrized. 
Under the parameterization, a Riemann surface S embedded in R 3 can be repre- 
sented by the position vector (j)(u,v): 

<j>(u, v) = (X(u, v), Y(u, v), Z(u, v)) e M 3 . (11) 

According to the Riemann surface theory, the surface S can be determined 
uniquely (up to a rigid motion) by its first fundamental form ds 2 and second fun- 
damental form. The first fundamental form can be written as follows: 



ds 2 = E(u, v)du 2 + 2F(u, v)dudv + G(u, v)dv 2 (12) 

where E{u,v) =< 4> u ,4>u >, F{u,v) =< (\) u ,<\>v > and E{u,v) =< (\) v ,(\) v >. 

If (j) is a conformal parameterization, F(u, v) = and E(u, v) = F(u, v) = \{u, v), 
where A is the conformal factor. Hence, 

ds 2 = \(u,v) 2 (du 2 +dv 2 ) (13) 
The second fundamental form can be written as follows: 



II = L(u, v)du 2 + 2M(u, v)dudv + N(u, v)dv 2 (14) 

where L(u,v) =< <fr U u,n >> M(u,v) =< (j) uv ,n > and N(u,v) =< (j) vv ,n > (ft is 
the unit surface normal). 

Now, our goal is to find geometric quantities which can determine the position 
vector (j) := (X, Y, Z) G M 3 . 

Let z = u + iv (i = \/— T), dz = du + idv, dz = du — idv, = §( — i J^y) and 

<9z — 2 Van ^ L dv)' 

Suppose (j) is a conformal parameterization, then we must have the following: 



< (j) z ,(j) z > = 

<<^,<fe> = , . 

A 2 

< > = —. 

Besides, the natural frame {(j) zi (j) zi n) must satisfy certain equations. Consider 
a =< 4> ZZl n >. Then, the natural frame satisfies: 
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(A) (B) (C) 

Figure 2. X-H representation of the human face. (A) shows a hu- 
man face. (B) shows the conformal factor A of the human face. (C) 
shows the mean curvature of the human face. The scalar functions 
conformal factor and mean curvature determine the surface up to 
a rigid motion. 

Clearly, the natural frame can be determined by the conformal factor A, mean 
curvature H and \i. Besides, /i can as well be determined by A and H through the 
Codazzi equation: 

Differentiate equation (17) with respect to z, we obtain 

Afi=^X(2X z H z + XH zz ). (18) 

Given the value of (j) on the boundary neighborhood U, the natural frame ((j) zi (j) zi n) 
can be found by solving the system of partial differential equation (16). The po- 
sition vector (j) can be reconstructed by integrating the natural frame. Therefore, 
the mean curvature H and conformal factor A are two geometric quantities fully 
determining a Riemann surface. Given a Riemann surface 5, its associated (A, H) 
can be computed. Conversely, given (A,i7), the associated Riemann surface S can 
be reconstructed. We call the pair (A, H) the X-H representation of S. 

The above discussion can be summarized by the following theorem. 

Theorem 4.1 (X-H representation). Let S be an open Riemann surface with mean 
curvature H . Let <j) : D C M 2 be the conformal parameterization of S with conformal 
factor X. Suppose U is a (small) boundary neighborhood of S. Given the boundary 
value <j)\u of <p, S is uniquely determined by X and H. 

5. Surface inpainting. In this section, we will describe a surface inpainting algo- 
rithm based on the X-H representation of the surface S. 

Suppose S is an incomplete 3D surface model with holes. Let £1 be the occluded 
region. Our goal is to obtain a restored surface S' with holes filled based on the 
surface geometry, so that the basic geometry of the surface can be smoothly patched. 
Following conventional image inpainting algorithms, we first take an initial surface 
5o with the holes roughly filled. Computationally, it can be done by creating a 
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Figure 3. X-H representation of a human tooth. (A) shows a hu- 
man tooth. (B) shows the conformal factor A of the tooth. (C) 
shows the mean curvature of the tooth. The scalar functions con- 
formal factor and mean curvature determine the surface up to a 
rigid motion. 




Figure 4. X-H representation of the human brain surface. (A) 
shows a human brain surface. (B) shows the conformal factor A 
of the human brain surface. (C) shows the mean curvature of the 
human brain surface. 



patch piecewise linearly with boundary vertices given by dD. So is then iteratively 
modified to obtain a satisfactory inpainted surface S f . 

One way to modify So is to update the position vector of So iteratively. However, 
this method does not consider the essential geometric information of the surface. 
We therefore propose to inpaint the surface by considering the conformal factor A 
and mean curvature H, which are the two essential geometric quantities determining 
a Riemann surface. 

Let 0o : Do C M? — » So be the conformal parameterization of So- We can 
compute the conformal factor Ao of </>o which is given by: 

A =<0n,0n> (19) 

Suppose Ho be the mean curvature of So- Then, (Ao, Ho) determines So- As 
expected, the scalar functions Ao and Ho are not smooth on the occluded region 
0q 1 (£l) in the conformal parameter domain. Our goal is to smooth out (or inpaint) 
Ao and Ho on the occluded region 0q x (^), while preserving the value of them on 
the non-occluded region Do \ </>q 1 (^)- 

The conformal factor and mean curvature are smooth scalar functions defined on 
the 2D conformal parameter domain. To ensure the smoothness of the inpainted 
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mean curvature H' and conformal factor A', the following inpainting model is ap- 
plied: 

A' = argmin A £;(A) := argmin A / |AA| 2 (20) 
subject to the constraint that: 

^liWo" 1 ^) = A oIa,\0o x (n) ^ 21 ) 
Also, we apply the same inpainting model for H'\ 

H' = argmin H E(H) := argmin^ / \AH\ 2 (22) 
subject to the constraint that: 

h '\d \4>o = H o\D \4>o\n) ( 23 ) 
By minimizing the variational models (20) and (22), we obtain an optimal (in- 
painted) conformal factor A' and H' . The surface associated with the pair (X f ,H f ) 
can then be reconstructed, as described in the last subsection, to obtain an inpainted 
surface S f . The surface holes will be effectively filled, with the basic geometry of 
the surface smoothly patched. 

The minimization of the variational problems (20) and (22) can be done itera- 
tively as follows: 

= -(A)'A(t) and ^ = -(A)'iT(i) (24) 
subject to A(t)| D(A0 - 1(n) = Aol^^-ijn) and H(t)\ Do ^-i {Q) = Ho\ Do \^ {Qy 

6. Numerical algorithms. In this section, we will describe how the proposed 
algorithms can be implemented. We will firstly describe how the X-H representation 
can be implemented. We will then explain how the surface inpainting algorithm can 
be implemented. 

6.1. Numerical implementation of X-H representation. Given a surface mesh 
5, the X-H representation of S can be computed easily using the conformal param- 
eterization of S. Many conformal parameterization algorithms have recently been 
developed and some of them are available online. We will not explain the conformal 
parameterizations in this paper, but refer the readers to [28, 29, 30, 32, 33, 34] for 
details. In this work, we use the algorithm proposed in [34] to obtain the conformal 
parameterization of S onto the rectangular conformal parameter domain R. 

Let (j)(u,v) := (X(u,v),Y(u,v), Z(u,v)) be the conformal parameterization of 
the surface mesh S. We consider the finite difference discretization of the partial 
derivatives on the rectangular conformal parameter domain R := [0, 1] x [0, K]. Let 
Ui = i/N, vj = Kj/N and h = 1/N. We denote faj = cj)(ui,Vj); Hij — H(iii,Vj); 
and Xij = X(ui,Vj). Using equation (12), the conformal factor A^ can be computed 
by: 



Ki = [ r^v^T + r ^ 2 ^-Y ]/2 (25) 

And using equation (10), the mean curvature H^ can be computed by: 
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Figure 5. Surface reconstruction of the human face from its X-H 
representation. (A) shows the original human face. The recon- 
structed human face from its X-H representation is shown in (B). 




(A) (B) 

Figure 6. Surface reconstruction of the human tooth from its A- 
H representation. (A) shows the original tooth. The reconstructed 
tooth surface from its X-H representation is shown in (B). 





Figure 7. Surface reconstruction of the human brain from its X-H 
representation. (A) shows the original brain surface. The recon- 
structed brain surface from its X-H representation is shown in (B). 



1 

where: 



H ij = 7T^ si g n (^j)l A( fel ( 26 ) 



2Xij 
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*+l,j + <t>i-l,j 



13 ~ ft 2 

sign(^) = sign(< A0^,n^- >) 



2ft 



2ft 



2ft 



2ft 



(27) 



Conversely, given the A-i7 representation of S, we can reconstruct the surface by 
solving the equation (16). Let (Uij,Vij,Wij) = ((4> z )ij, {^>z)ij^ij) be the natural 
frame at (ui,Vj). Consider the central difference discretization of and J| as 
follows: 



df_ 

dz 

dj_ 

dz 



fi+l,j fi- 

2ft 

fi+l,j ~ fi — l,j 

2ft 



2ft 



2ft 



(28) 



Hence, (\ z )ij can be computed for all < i, j < iV. Together with the boundary 
conditions, the natural frame (Uij,Vij,Wij) can then be computed by solving the 
following linear system, which approximates the solution of the equation (16): 



1 

2ft 



Ui-ij) 
■Vi-ij) 



2hWa 



A 2 

TT TT _ 2fIijVjj 



(29) 

Once the natural frame is obtained, the position vector <pij (and hence the surface 
mesh S) can be reconstructed by solving the linear system (with the given boundary 
conditions): 




(Uij + Vij)/2 



(30) 



6.2. Numerical implementation of surface inpainting. Given a surface mesh 
with holes 5, we first obtain an initial mesh So with the holes roughly filled. It can 
be obtained by the delaunay triangulation of the boundary vertices of the surface 
holes. The X-H representation of So can then be computed, which are two non- 
smooth scalar functions of the conformal factor A and mean curvature H°. We 
then inpaint (or smooth out) Ao and Ho at the occluded regions on the conformal 
parameter domain, using the inpainting models (20) and (22). 

Consider the discretization of the Laplacian on the conformal parameter domain 
as follows: 



(A/)« = 



fi+lj + fi-lj ~ ^fij + fi,j+l + fij-l 



H3 - h 2 

With this discretization, the inpainting models on A and H° can be implemented 
as follows: 
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(A) (B) (C) 




Figure 8. Surface inpainting of the cubic surface. (A) shows the 
cubic surface with an occlusion. (B) shows the inpainted sur- 
face obtained from the conventional smooth hole-filling method. 
(C) shows the inpainted surface using our proposed conformal ap- 
proach. (D) and (E) shows the initial conformal factor and in- 
painted (smoothed) conformal factor respectively. (F) and (G) 
shows the initial mean curvature and inpainted (smoothed) mean 
curvature respectively. 



Hij = Hif 1 + dt[A(AH n ~ 1 )]ij 

with the constraints that A™- = A^- and = H^- for all (u^ vj) that are not in the 
occluded regions, dt is the time step. 

7. Experimental results. To examine the effectiveness of the proposed method, 
we have tested our algorithms on synthetic data together with real surface data. In 
this section, we will describe the experimental results in details. 

7.1. Surface reconstruction from X-H representation. We first examine how 
well the X-H representation can represent a Riemann surface. In Figure 2, we 
compute the conformal factor A and mean curvature H of a human face. This 
gives the X-H representation of the human face. Using X-H representation, we can 
accurately reconstruct the original surface. The reconstructed surface is shown in 
Figure 5(B), which closely resembles to the original surface as shown in Figure 5(A). 
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Figure 9. Surface inpainting of the human tooth surface. (A) 
shows the tooth surface with an occlusion. (B) shows the in- 
painted surface obtained from the conventional smooth hole-filling 
method. (C) shows the inpainted surface using our proposed con- 
formal approach. (D) and (E) shows the initial conformal factor 
and inpainted (smoothed) conformal factor respectively. (F) and 
(G) shows the initial mean curvature and inpainted (smoothed) 
mean curvature respectively. 



We also compute the X-H representation of a human tooth, as shown in Figure 
3. From the X-H representation, the tooth surface can be effectively and accurately 
reconstructed. Figure 6(B), shows the reconstructed tooth surface from its X-H 
representation, which again closely resembles to the original tooth surface as shown 
in Figure 6(A). 

Finally, we compute the X-H representation of a human brain, whose geometry 
is complicated. The conformal factor and mean curvature of the brain surface is 
shown in Figure 4. From the X-H representation, the brain surface can be recon- 
structed as shown in Figure 7. The reconstructed surface is close to the original 
one. It demonstrates that the proposed geometric representation is effective, even 
for surfaces with complicated geometry. 
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(A) (B) (C) 




(D) Conformal factor (E) 




Figure 10. Surface inpainting of the snowman surface. (A) shows 
the snowman surface with two holes. (B) shows the inpainted sur- 
face obtained from the conventional smooth hole-filling method. 
(C) shows the inpainted surface using our proposed conformal ap- 
proach. (D) and (E) shows the initial conformal factor and in- 
painted (smoothed) conformal factor respectively. (F) and (G) 
shows the initial mean curvature and inpainted (smoothed) mean 
curvature respectively. 

7.2. Surface inpainting. Next, we test the surface inpainting algorithm on syn- 
thetic data and real surface data. In Figure 8, we test the algorithm on a cubic 
surface with a hole. To inpaint the surface, a conventional way is to smoothly 
fill in the surface hole with its boundary representation in 3D [23, 22]. Although 
the surface can be smoothly filled, the inpainted surface cannot follow the surface 
geometry. As shown in (B), the inpainted surface generated by the conventional 
smooth hole-filling algorithm cannot preserve the sharp edge of the cubic surface. 
The sharp edge is cropped off. (C) shows the inpainted surface using our proposed 
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(F) (G) 

Figure 11. Surface inpainting of the human face surface. (A) 
shows the human face with three holes. (B) shows the inpainted 
surface obtained from the conventional smooth hole-filling method. 
(C) shows the inpainted surface using our proposed conformal ap- 
proach. (D) and (E) shows the initial conformal factor and in- 
painted (smoothed) conformal factor respectively. (F) and (G) 
shows the initial mean curvature and inpainted (smoothed) mean 
curvature respectively. 

conformal approach. The inpainted surface can better preserve the sharp edge. 
(D) and (E) shows the initial conformal factor and inpainted (smoothed) conformal 
factor respectively. (F) and (G) shows the initial mean curvature and inpainted 
(smoothed) mean curvature respectively. By inpainting the mean curvature and 
conformal factor, the inpainted surface follows the surface geometric pattern of the 
non-occluded regions of the surface. 

In Figure 9, we examine the proposed inpainting algorithm on a human tooth 
surface. (A) shows a tooth surface with an occlusion. (B) shows the inpainted 
surface obtained from the conventional smooth hole-filling algorithm. Again, the 
inpainted surface does not follow the surface geometry of the non-occluded regions 
of the surface. (C) shows the inpainted surface generated by our proposed inpainting 
method. The inpainted surface is more natural which follows the surface geome- 
try. The basic geometry can be smoothly patched. (D) and (E) shows the initial 
conformal factor and inpainted (smoothed) conformal factor respectively. (F) and 





Conformal factor 





Figure 12. Surface inpainting of the human brain surface. (A) 
shows the original brain surface. We remove four regions on the 
brain surface, as shown in (B). (C) shows the inpainted surface 
using our proposed conformal approach. Note that the inpainted 
patches closely resemble to the original one. (D) and (E) shows 
the initial conformal factor and inpainted (smoothed) conformal 
factor respectively. (F) and (G) shows the initial mean curvature 
and inpainted (smoothed) mean curvature respectively. 



(G) shows the initial mean curvature and inpainted (smoothed) mean curvature 
respectively. 

In Figure 10, we show the surface inpainting result of a snowman surface. (A) 
shows the snowman surface with 2 holes. (B) shows the inpainted surface obtained 
from the conventional smooth hole-filling algorithm. The inpainted surface patches 
are flatten and the inpaint surface does not follow the surface geometry. (C) shows 
the inpainted surface using our method. The inpainted surface is more natural 
which follows the surface geometry. 

Figure 11 shows the example of surface inpainting of a human face. (A) shows 
a human face with 3 holes, one on the nose and two on the cheeks. (B) shows 
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the inpainted surface obtained from the conventional smooth hole-filling algorithm. 
The inpainted surface is not natural and does not following the surface geometry. 
For example, the inpainted surface patch (inside the box) for the hole on the nose is 
flatten. (C) shows the inpainting result obtained by our algorithm. The reconstruct 
surface is more natural and follows the surface geometric pattern. 

Finally, in Figure 12, we test the inpainting algorithm on a brain surface, which 
has more complicated geometry. (A) shows the original brain surface. We remove 
four regions on the brain surface, as shown in (B). The inpainted surface generated 
by our algorithm is shown in (C). Note that the inpainted surface patches closely 
resemble to the original ones (compare with (A)). This example demonstrates that 
our proposed inpainting algorithm perform well even on surfaces with complicated 
geometry. 



8. Conclusion and future works. In this work, we propose a novel conformal 
approach for surface inpainting to fill in missing holes on an incomplete 3D surface 
model. It has important applications in different fields, such as in medical imaging, 
computer graphics and computer visions. In order to fully manipulate the surface 
geometry of the captured 3D model, we consider in this work a new representation 
of 3D surfaces based on their conformal factor A and mean curvature H. According 
to Riemann surface theory, a Riemann surface can be uniquely determined by its 
conformal factor and mean curvature up to a rigid motion. Given a Riemann 
surface 5, its mean curvature H and conformal factor A can be computed easily 
through its conformal parameterization. Conversely, given A and H, a Riemann 
surface can be uniquely reconstructed by solving the Gauss-Codazzi equation on the 
conformal parameter domain. Hence, the conformal factor and the mean curvature 
are two geometric quantities fully describing the surface. In this paper, we develop 
a numerical algorithm to reconstruct surfaces from their conformal factor and mean 
curvature by solving systems of linear equations. Given the X-H representation of an 
incomplete surface, the problem of surface inpainting can be reformulated into the 
image inpainting problem of A and H on the conformal parameter domain. Using 
the conventional image inpainting techniques, both A and H can be inpainted on the 
parameter domain. A Riemann surface can then be reconstructed which restores the 
3D surface with surface holes. Since the inpainting model is based on the geometric 
quantities A and H, the restored surface follows the surface geometric pattern as 
much as possible. We test the proposed algorithm on synthetic data, 3D human face 
data and MRI-derived brain surfaces. Experimental results show that our algorithm 
can effectively inpaint surface holes based on the geometry of the surface to restore 
the incomplete 3D surface models. 

Besides, we observe from our experiments that the X-H representation of a surface 
is a useful geometric representation of a surface mesh capturing the most essential 
geometric information. By manipulating A and H, the surface mesh can be edited 
based on the surface geometry. Thus, we believe there are several potential applica- 
tions of this representation to other surface mesh editing problems, such as surface 
mesh compression, surface mesh denoising and so on. In the future, we will consider 
applying the X-H representation for surface stitching. We will also consider extend- 
ing the X-H representation to point cloud. With that, point cloud compression, 
point cloud inpainting and point cloud denoising can be easily achieved. 
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